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System and method for simulating fill flash In photography 



THE ASSISTANT COMMISSIONER OF PATENTS 
Washington, D.C. 20231 

Sir; 

An appeal brief was previously filed in this case on August 17, 2005. That previous 
brief was deemed defective for failing to comply with a provision of 37 CFR 41 .37. 
Applicant regrets the inconvenience caused by the noncompliant brief, and encloses a revised 
brief herewith. 
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BRIEF ON APPF.AT. 
INTRODUCTION 

Pursuant to the provisions of 37 CFR § 41.30 et seg., applicants hereby appeal to the 
Board of Patent Appeals and Interferences from the examiner's rejection dated 07/15/2005. 
The claims in question have been finally rejected. 

REAL PARTY IN INTEREST 

The entire interest in the present application has been assigned to Hewlett-Packard 
Development Company, L.P., as recorded at Reel 014061, Frame 0492. 

RELATED APPEALS AND INTERFERENCES 
There are no related appeals or interferences. 

STATUS OF CLAIMS 
Claims 1-24 are pending in the application. 

Claims 1-24 stand finally rejected, and are the subject of this appeal. 

STATUS OF AMENDMENTS 
No amendments have been filed after the final rejection mailed July 15, 2005. 

SUMMARY OF CLAIMED SUBJECT MATTER 

Claim 1 is directed to a method of simulating fill flash in a camera system, which is 
described in the specification at least at page 3 lines 14-21, page 10 lines 16-24, page 1 1 lines 
3-5, page 1 1 line 23 through page 12 line 22, and Figure 9. The method includes determining 
distances from a camera to objects in a scene (901), taking a photograph without using a flash 
(908), and selectively adjusting the brightness or regions in the photograph based on the 
distance information (909). 

Claim 7 is directed to a camera system that simulates fill flash. The camera system is 
described in the specification at least at page 4 line 17 through page 6 line 12, and in Figure 1 . 
The camera system simulates fill flash by performing the steps of the method of claim 1. 

Claim 14 is directed to a camera, which is described in the specification at least at 

HP Docket No. 10992614-1 page 2 of 15 Serial No. 09/955.457 

PAGE 3/16 * RCVD AT 4/1812006 12:02:26 PM [Eastern DayUght rime] ' SVfcUSPTO-EFXRF-6/43 ' ONS:2738300 ' CSID:9708987247 ' DURATION (mnKS):0340 



APR-18-2006 10:04 HP LEGRL FT COLLINS 9708987247 P. 04 



page 4 line 17 through page 6 line 12, and in Figure 1. The camera comprises means for 
performing the steps of the method of claim 1 for simulating fill flash. 

GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

1. Whether claims 1, 7, and 14 are unpatentable under 35 U.S.C. § 103(a) over 
Nishimura et al. (U.S. Pat. No. 5,617,141), 

2. Whether claims 2-4, 8-1 1, 23, and 24 are unpatentable under 35 U.S.C. § 103(a) over 
Nishimura et al. (U.S. Pat. No. 5,617,141) in view of Parulski et al. (U.S. Pat. No. 
5,563,658). 

3. Whether claims 5, 6, 12, and 13 are unpatentable under 35 U.S.C. § 103(a) over 
Nishimura et al. (U.S. Pat No. 5,617,141) in view of Miyadera (U.S. Pat. No. 
5,550,587). 

4. Whetherclaims 15-22 are unpatentable under 35 U.S.C. § 103(a) over Nishimura et 
al. (U.S. Pat. No. 5,617,141) in view of Kikuchi (U.S. Pat. No. 6,757,020). 

ARGUMENTS 

1 . ARGUMENT: Rejection of claims 1, 7, and 14 under 35 U.S.C. § 103(a) 

The examiner has rejected claims 1, 7, and 14 as being unpatentable under 35 U.S.C 
§ 103(a) over Nishimura et al. (U.S. Pat. No. 5,617,141). Applicant respectfully submits that 
the rejection is improper because the examiner has not made out zprima facie case of 
obviousness. 

A. Applicable law 

To establish a prima facie case of obviousness, three basic criteria must be met. First, 
there must be some suggestion or motivation, either in the references themselves or in the 
knowledge generally available to one of the ordinary skill in the art, to modify the reference 
or to combine reference teachings. Second, there must be a reasonable expectation of 
success. Finally, the prior art reference (or references when combined) must teach or suggest 
all me claim limitations. MPEP2143. 

The teaching or suggestions to make the claimed combination and the reasonable 
expectation of success must both be found in the prior art, not in the applicant's disclosure. 
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MPEP 2143 citing/* re Vaeck, 947 F.2d 488, 20 USPQ2d 1438 (Fed. Cir. 1991). 

If an independent claim is nonobvious under 35 U.S.C. 103, then any claim 
dependent therefrom is nonobvious. In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed Cir 
1988). 

Without conceding any other elements required to establish a prima fade case of 
obviousness, Applicant respectfully submits that Nishimura et al. do not teach or suggest all 
the limitations of Applicant's claims 1, 7, and 14. 

B. Claim 1 

Claim 1 recites a method comprising in part selectively adiucHng the hrip ht^cc »f 
regions of the photograph h ased on th e distance i^ m3 (i m 

Nishimura et al. do not teach this claim element. In support of the rejection, the 
examiner cites column 5 line 52 through column 7 line 1 7 of Nishimura et al. While part of 
the cited passage does describe exposure control, no mention is made of selectively aHinst^ 
the brightness of regions of fh* photog ra ph. The system of Nishimura et al. uses a different 
"detection characteristic" to set exposure depending on scene "ambience". (Nishimura et al 
column 7 lines 14-17.) Exposure is controlled using a "stop" (Nishimura et al. co IU mn 2 lines 
10-14 and column 3 lines 48-49) or an exposure time (column 4 lines 5-9). As is well 
known, a controlling exposure using either a stop, or an exposure time, or both affects the 
exposure of an entire photograph substantially uniformly. Different regions of the photograph 
are not affected selectively by Nishimura et aL 

Applicant respectfully notes that the "regions" depicted in Figure 4 of Nishimura et al 
are not regions of a photograph. Nishimura's Figure 4 is not a map of a scene or photograph, 
but is an abstract construct for categorizing a particular scene based on measurements of the 
subject mumination and a background distance. A particular location in the "map" of 
Nishimura et al. does not correspond to a particular location in a scene, but indicates a 
characteristic of the entire scene. (Nishimura et al. column 5 lines 18-51, column 2 lines 54- 
62, and Figure 4.) 

C. Claim 7 

Claim 7 is an apparatus claim to a camera system that simulates fill flash by 
performing the steps of the method of claim 1. Because, as is shown above, Nishimura et al. 
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do not teach or suggest all of the limitation, of Applicant's claim 1, they similarly do not 
teach or suggest all of the limitations of Applicant's claim 7. 

D. Claim 14 

Claim 14 is an apparatus claim to a camera that comprises means for performing the 
steps of the method of claim 1. Because, as is shown above, Nishhnura et al. do not teach or 
suggest all of the limitations of Applicant's claim 1, they similarly do not teach or suggest all 
of the limitations of Applicant's claim 14, 

Because Nishhnura et al. do not teach or suggest all of the limitations of Applicant's 
claims 1, 7, and 14, the examiner's prima facie case of obviousness fails. Reversal of the 
rejection is respectfully requested. 

2. ARGUMENT: Rejection of claims 2-4, 8-11, 23, and 24 under 35 U.S.C. § 103(a) 
The examiner has rejected claims 2A, 8-1 1, 23, and 24 as being unpatentable under 35 
U.S.C. § 103(a) over Nishhnura et al. (U.S. Pat. No. 5,617,141) in view of Parulski et al 
(U.S. Pat No. 5,563,658). Applicant respectfully submits that the rejection is improper 
because the examiner has not made out iprima facie case of obviousness. Without 
conceding any other elements required to establish a prima facie case of obviousness 
Applicant respectfully submits that the combination of Nishimura et al. and Parulski et al. 
does not teach or suggest all the limitations of these claims. 

A. Claims 2-4 and 23 

Applicant's claims 2-4 and 23 depend from claim 1 and add further limitations. 
Claun 1 is shown above to be patentable over Nishimura et al. because Nishimura et al do 
not teach or suggest selectively adjusting the bri^, of me _ _ 

the distance information . Parulski et al. do not supply this element, and therefore the 
combination of Nishimura et al. and Parulski et al. does not teach or suggest all of the 
limitations of claims 2-4 and 23. 

B. Claims 8-11 and 24 

Applicant's claims 8-H and 24 depend from claim 7 and add further limitations. 
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Claim 7 is shown above to be patentable over Nishimura et al. because Nishimura et al. do 
not teach or suggest selectively adjusting the bright^ of region, of the p hotograph h* Re * on 
fe e distance information . Parulski et al. do not supply this element, and therefore the 
combination of Nishimura et al. and Parulski et aL does not teach or suggest all of the 
limitations of claims 8-1 1 and 24. 

Because Nishimura et al. and Parulski et al. in combination do not teach or suggest all 
of the limitations of Applicant's claims 2-4, 8-U, 23, and 24, the examiner's prima facie case 
of obviousness fails. Reversal of the rejection is respectfully requested. 

3. ARGUMENT: Rejection of claims S, 6, 12, and 13 under 35 U.S.C. § 103(a) 
The examiner has rejected claims 5, 6, 12, and 13 as being unpatentable under 35 
U.S.C. § 103(a) over Nishimura et al. (U.S. Pat. No. 5,617,141) in view ofMiyadera (U.S. 
Pat. No. 5,550,587). Applicant respectfully submits that the rejection is improper because the 
references used in the rejection are improperly combined, and because even if the references 
are combined, the examiner has not made out a prima fade case of obviousness. 

A. Improper combination of references 

It is improper to combine references where the references teach away from their 
combination. MPEP 2145 citing In re Grasselli, 713 F.2d 731, 743, 218 USPQ 769, 779 
(Fed. Cir. 1983) 

Applicant's claims 5 and 6 recite in part adjusting tha brightness of raring 0 f a 
photograph (a limitation present in Applicant's claims 5 and 6 by virtue of their dependence 
on claim 1). Claims 12 and 13 recite in part a system that lightens re^™ 
closer to the camera jn , the reacting photograph in rritinn to regions rnnt*^ .Kj~, g 
farmer from ^carngm . As is described at column 7 line 65 through column 8 line 43 of 
Miyadera, Miyadera's method of adjusting white balance involves computing gains based on 
a weighted average of the gains used for performing white balance under pure flash 
illumination (AR1 and ABl) and under pure ambient daylight (AR2 and AB2), Applicant 
notes that in all of the examples given, the weightings of these gains sum to unity, apparently 
so that white balance is adjusted without affecting brightness. In other words, Miyadera 
endeavors to preserve brightness rather than adjust it. Because Miyadera teaches away from 
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applicant's method, the combination of Miyadera with other art in establishing obviousness is 
improper. 



facie case 



B. Even if combined, the referenced do not support a prima fa 

Without conceding any other elements required to establish a prima facie case of 
obviousness, Applicant respectfully submits that the combination ofNishimura et al. and 
Miyadera does not teach or suggest all the limitations of claims 5, 6, 12, and 13. 

i. Claims 5 and 6 

Applicant's claims 5 and 6 depend from claim 1 and add further limitations. Claim 1 
is shown above to be patentable over Nishimura et al. because Nishimura et al. do not teach 
0f SUggeSt selectively adjustin g the b rightne ss of regions of the photograp h m 
distance information , Miyadera does not supply this element, and therefore the combination 
ofNishimura et al. and Miyadera does not teach or suggest all of the limitations of claims 5 
and 6. 

Furthermore, claims 5 and 6 recite in part the method of claim 1 wherein regions 
containing objects closer to the camera are lightened in th e reS ulW p Wnp raph in tn 
regions containing objects farther from the camera . The examiner relies on Miyadera to teach 
this claim limitation, citing column 14 lines 1-6 of Miyadera. The cited passage does not 
support the rejection. The cited passage describes "controlling a coefficient" used in "white 
balance adjustment" based on "image-object distance". Applicant first notes that white 
balance adjustment is not the same as liehtenrinp] regions in a photograph. 

And, as is explained above, Miyadera endeavors to preserve brightness, not adjust it, 
and therefore does not describe a method in which regions containin g objects to th. 

camera are lightened in the resulting photograp h i n relation to rep i ons containing ohj^ 
farther from the camwa 

ii. Claims 12 and 13 

Applicant's claims 12 and 13 depend from claim 7 and add further limitations. Claim 
7 is shown above to be patentable over Nishimura et al. because Nishimura et al. do not teach 
0r SUg 8 est selectively adjusting the brig htn ess of regions of the photograp h ^ rtu. 

dis tance information . Miyadera does not supply this element, and therefore the combination 
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of Nishimura et al. and Miyadera does not teach or suggest all of the limitations of claims 12 
and 13. 



even 



Because Nishimura et al. and Miyadera are improperly combined, and because 
when combined they do not teach or suggest all of the limitations of Applicant's claims 5, 6, 
12, and 13, the examiner's prima facie case of obviousness fails. Reversal of the rejection is 
respectfully requested. 

4. ARGUMENT: Rejection of claims 15-22 under 35 U.S.C. § 103(a) 
The examiner has rejected claims 15-22 as being unpatentable under 35 U.S.C. § 
103(a) over Nishimura et al. (U.S. Pat. No. 5,617,141) in view of Kikuchi (U.S. Pat. No. 
6,757,020). Applicant respectfully submits that the rejection is improper because the 
examiner has not made out zprima facie case of obviousness. Without conceding any other 
elements required to establish a prima facie case of obviousness, Applicant respectfully 
submits that the combination of Nishimura et al. and Kikuchi does not teach or suggest all the 
limitations of these claims. 

A. Claims 15-18 

Applicant's claims 15-18 depend from claim 1 and add further limitations. Claim 1 is 
shown above to be patentable over Nishimura et al. because Mshimura et al. do not teach or 
SUggCSt selectively adjusting the brightness of raions nf ih» p hotograph based on the distent 
informat io n . Kikuchi does not supply this element, and therefore the combination of 
Nishimura et al. and Kikuchi does not teach or suggest all of the limitations of claims 1 5-1 8. 

B. Claims 19-22 

Applicant's claims 19-22 depend from claim 7 and add further limitations. Claim 7 is 
shown above to be patentable over Nishimura et al. because Nishimura et al. do not teach or 
suggest selectively adjusting the bn>hmess of regions nf t he nnotopranh on the distant 

information . Kikuchi does not supply this element, and therefore the combination of 
Nishimura et al. and Kikuchi does not teach or suggest all of the limitations of claims 19-22. 

Because Nishimura et al. and Kikuchi in combination do not teach or suggest all of 
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the limitations of Applicant's claims 15-22, the examiner's prima facie case of obviousness 
fails. Reversal of the rejection is respectfully requested. 

Conclusion 

In view of the above, applicant respectfully requests that all of the examiner's claim 
rejections be reversed. 

Respectfully submitted, 



April 7, 2006 

Fort Collins, CO 80525 

(970) 898-4475 



By Kiuj/d^uJ, Jkr^ A^ 



David W. Boyd 
Reg. Number 50,335 
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CLAIMS APPENDIX 

I. A method of simulating fill flash in a camera system comprising the steps of: 
detennining distances from the camera to objects in a scene; 
taking a photograph of the scene without using a flash; and 

selectively adjusting the brightness of regions of the photograph based on the distance 
information- 

2. The method of claim 1 wherein determining the distances from the camera to objects in 

the scene comprises; 

taking a series of photographs with the camera configured to focus on objects at 

various distances from the camera; and 
analyzing the series of photographs and corresponding focus distances, 

3. The method of claim 2 wherein the analysis of the series of photographs comprises 
computing a spatial contrast metric. 

4. The method of claim 3 wherein detennining the distances to objects at locations in the 
scene further comprises: 

locating the particular photograph in the series of photographs with the spatial contrast 
metric indicating that objects at that location in the scene » are more nearly in 
focus in that particular photograph than in any other in the series of 
photographs; and 

identifying the distance from the camera to objects at that location in the scene as the 
focus distance stored in connection with that particular photograph. 

5. The method of claim 1 wherein regions containing objects closer to the camera are 

lightened in the resulting photograph in relation to regions containing objects farther from 
the camera. 
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6. The method of claim 5 wherein regions are modified in the resulting photograph in 
accordance with the inverse square law. 

7. A camera system which simulates fill flash by: 

determining distances from the camera to objects in a scene; and 
taking a photograph of the scene without using a flash; and 

selectively adjusting the brightness of regions of the photograph based on the 

distances. 

8. The camera system of claim 7 wherein the determining the distances from the camera to 
objects in the scene comprises: 

taking a series of photographs with the camera configured to focus on objects at 
various distances from the camera; and 

analyzing the series of photographs and corresponding focus distances to determine 
the object distances. 

9. The camera system of claim 8, further comprising a computer separate from the camera, 
and wherein the series of trial photographs and their focus distances are transmitted to the 
separate computer for analysis and the simulation of fill flash. 

10. The camera system of claim 8 wherein the analysis of the series of photographs 
comprises computing a spatial contrast metric. 

H . The camera system of claim 8 wherein detemjining the distances to objects at locations in 
the scene further comprises: 

locating the particular photograph in the series of photographs with the spatial contrast 
metric indicating that objects at that location in the scene are more nearly in 
focus in that particular photograph than in any other in the series of 
photographs; and 

identifying the distance from the camera to objects at that location in the scene as the 
camera focus distance stored in connection with that particular photograph. 
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12. The camera system of claim 7 wherein the system lightens regions containing objects 
closer to the camera in the resulting photograph in relation to regions containing objects 
farther from the camera, 



13- The camera system of claim 12 wherein the system modifies the brightness of regions 
the resulting photograph in accordance with the inverse square law. 



m 



14. A camera* comprising: 

means for determining distances from the camera to objects in a scene; 
means for taking a photograph without using a flash; and 

means for selectively modifying the brightness of regions in the resulting photograph 
based on the distances. 

15. The method of claim 1, further comprising varying the amount of brightness adjustment 

in response to a user control. 



16. The method of claim 15, wherein the user control is comprised in the 



camera. 



17. The method of claim 15, further comprising displaying the effect of the brightness 

adjustment on a display. 

1 8. The method of claim 17, wherein the display is comprised in the camera. 

19. The camera system of claim 7, wherein the camera system further simulates fill flash by 

varying the amount of brightness adjustment in response to a user control. 



20. The camera system of claim 19, wherein the user control is comprised in the 



camera. 
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21. The camera system of claim 7, wherein the camera system further simulates fill flash by 

displaying the effect of the brightness adjustment on a display. 

22. The camera system of claim 21, wherein the display is comprised in the camera. 

23. The method of claim 2, wherein one of the series of photographs taken in determining the 

distances from the camera to objects in the scene is used as the photograph in which 
the brightness of regions is selectively adjusted. 

24. The camera system of claim 8> wherein one of the series of photographs taken in 

determining the distances from the camera to objects in the scene is used as the 
photograph in which, the brightness of regions is selectively adjusted. 
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EVIDENCE APPENDIX 

None. 
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RELATED PROCEEDINGS APPENDIX 

None, 
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